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Social Skills Training 

Program Description^ 

Social skills training is not a specific curriculum, but rather a collection 
of practices that use a behaviorai approach for teaching preschool 
children age-appropriate social skills and competencies, including 
communication, probiem solving, decision making, self-management, 
and peer relations. Social skills training can occur in both regular and 
special education classrooms. 

A variety of social skills training approaches and curricula are available. 

For example, teachers may use a structured approach to explain to 
students how to enact a desired behavior by providing examples and 
reinforcing targeted behaviors through questions, answers, and other 
feedback. An example of a more nuanced approach (often referred to 
as “incidental teaching”) is when teachers respond to student-gener- 
ated utterances, interactions, and behavior to encourage the desired 
social skills (such as rewarding positive play). 

Research^ 

The What Works Clearinghouse (WWC) identified three studies of 
social skills training that both fall within the scope of the Early Child- 
hood Education Interventions for Children with Disabilities topic area 

and meet WWC evidence standards. All three of these studies meet standards without reservations and together, they 
included 135 children with disabilities in early education settings in the United States. Although this report presents 
information about all three studies and their findings, the WWC’s summary ratings of the evidence of effectiveness of 
the intervention are based on only two of the studies, that, together, included 1 03 children in their samples. The third 
study, which had a sample of 32 children, did not provide sufficient information to support calculation of effect sizes 
and statistical significance, which are used in determining the WWC’s overall evidence ratings. 
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The WWC considers the extent of evidence for social skills training on children with disabilities in early education 
settings to be small for two outcome domains— (a) cognition and (b) social-emotional development and behavior. 
There were no studies that meet standards in the five other domains, so we do not report on the effectiveness of 
social skills training for those domains in this intervention report. (See the Effectiveness Summary on p. 5 for further 
description of all domains.) 


Effectiveness 

Social skills training was found to have no discernible effects on cognition and positive effects on social-emotional 
development and behavior for children with disabilities in early education settings. 
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Table 1. Summary of findings^ 




Improvement index 

(percentile points) 




Outcome domain 

Rating of effectiveness 

Average 

Range 

Number of 
studies 

Number of 
students 

Extent of 
evidence 

Cognition 

No discernible effects 

+7 

na 

1 

65 

Small 

Social-emotional 
development and behavior 

Positive effects 

+18 

-18 to +44 

2 

103 

Small 


na = not applicable 
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Program Information 

Background 

Social skills training does not have a single developer responsible for providing infornnation or materials. The 
interventions described in this report were developed by various study authors and are not available for distribution 
through a common developer. However, many online resources are available for readers interested in using social 
skills training practices. The following sites provide a general overview of social skills training methods: 

• Behavior Advisor: Teaching Social Skills: http://www.behavioradvisor.com/SocialSkills.html 

• Social Skills Training Project: http://www.socialskillstrainingproject.com/ 

• Social Work Podcast: http://socialworkpodcast.blogspot.eom/2010/06/social-skills-training-interview-with.html 

Program details 

Teachers can use social skills training practices with individual children, in small-group settings, or with whole 
classes. Regardless of the setting, social skills training practices are intended to promote positive interactions 
among children and between children and their teachers. In order to implement the social skills training approach, 
teachers use modeling, role-playing, and specific instruction on social skills. Then, children typically practice the 
skills and receive positive reinforcement for engaging in appropriate social behavior. Training and practice activi- 
ties typically occur for up to one hour each day. The duration of an intervention can vary from a few days to several 
weeks. More detailed practices for specific social skills training programs are presented in Appendices A.1 to A.3. 


Cost 

Some published social skills training programs are freely available to the public. The WWC was unable to identify 
information about the costs of implementing the intervention (e.g., for teacher training and support). 
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Research Summary 


The WWC identified 46 studies that investigated the effects oi social 
skills training on children with disabiiities in early education settings. 

The WWC reviewed three of those studies against group design 
evidence standards. All three of those studies (Ferentino, 1 991 ; 
Guglielmo & Tryon, 2001 ; Leblanc & Matson, 1995) are randomized 
controlied trials that meet WWC group design evidence standards 
without reservations and are summarized in this report. 

The WWC reviewed three additionai studies against the pilot single- 
case design standards. All three studies do not meet WWC pilot 
single-case design standards. The remaining 40 studies do not meet 
WWC eligibility screens for review in this topic area. (Citations for all 
46 studies are in the References section, which begins on p. 7). 


Table 2. Scope of reviewed research 


Grade 

Preschool 

Delivery method 

Whole class 

Program type 

Practice 

Studies reviewed 

46 

Group design studies 
that meet WWC 
evidence standards 

• without reservations 

• with reservations 

3 studies 
0 studies 


Summary of studies meeting WWC evidence standards without reservations 

Ferentino (1991) randomly assigned classrooms to one of three conditions, two of which included a social skills 
training curriculum called “My Friends and Me” and one of which was a wait-list control condition. This WWC report 
focuses on the 32 children in four classrooms that were assigned to a school-based implementation of the social 
skills training intervention and the 33 children in four classrooms that were assigned to the wait-list control condi- 
tion and did not receive the intervention during the study period.^ 

Guglielmo and Tryon (2001) randomly assigned a total of nine classrooms to receive various combinations of social 
skills training using the “Taking Part: Introducing Social Skills to Children” curriculum. This WWC report focuses on 
a subset of six classrooms included in the study. Nineteen children in three classrooms received the social skills 
training intervention, supplemented by a reinforcement of target behaviors. Nineteen children in three other class- 
rooms received the reinforcement of target behaviors without social skills training and serve as the comparison 
group to determine the effect of social skills training in this WWC report.® 

LeBlanc and Matson (1995) randomly assigned six classrooms to receive an unnamed social skills training cur- 
riculum or to a business-as-usual comparison condition. Sixteen children in three of the classrooms received social 
skills training, and 16 children in the other three classrooms did not. 

Summary of studies meeting WWC evidence standards with reservations 

No studies of social skills training meet WWC evidence standards with reservations. 
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Effectiveness Summary 

The WWC reviews of evidence for the Early Childhood Education Interventions for Children with Disabilities topic 
area addresses student outcomes in seven domains: (a) cognition, (b) communication/language competencies, (c) 
literacy, (d) mathematics achievement, (e) social-emotional development and behavior, (f) functional abilities, and (g) 
physical well-being. The three studies of social skills training that meet WWC evidence standards reported findings 
in two of the seven domains: (a) cognition and (b) social-emotional development and behavior. The findings below 
present the authors’ estimates and WWC-calculated estimates of the size and statistical significance of the effects 
of social skills training on children with disabilities in early education settings.® For a more detailed description of 
the rating of effectiveness and extent of evidence criteria, see the WWC Rating Criteria on p. 26. 

Summary of effectiveness for the cognition domain 

One study reported findings in the cognition domain. 

Ferentino (1991) reported findings that were not statistically significant or the one measure assessed in this out- 
come domain, and the WWC confirmed this calculation. The direction of this effect favored the social skills training 
group but was not large enough to be considered substantively important by WWC criteria. In this study, the effect 
was classified as indeterminate. 

Thus, for the cognition domain, one study reported findings that were not statistically significant or substantively 
important. This results in a rating of no discernible effects, with a small extent of evidence. 


Table 3. Rating of Effectiveness and extent of evidence for the cognition domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

None of the studies shows 
a statistically significant or 
substantively important effect, 
either positive or negative. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the cognition 
domain ms neither statistically significant nor large enough to be substantively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 65 children in one school reported evidence of effectiveness in the cognition domain. 


Summary of effectiveness for the social-emotional development and behavior domain 

Three studies reported findings in the social-emotional development and behavior domain. 

Ferentino (1991) analyzed five measures in this outcome domain. The study reported a statistically significant 
positive effect for one measure, which was confirmed by the WWC. The results for the other four outcomes were 
not statistically significant. According to WWC criteria, this study shows a statistically significant positive effect for 
social-emotional development and behavior.^ 

Guglielmo and Tryon (2001) examined impacts on two outcomes in this domain. For both outcomes, the estimated 
effects were positive. One of the findings was statistically significant and the other was not, but was large enough 
to be considered substantively important. According to WWC criteria, this study shows a statistically significant 
positive effect for social-emotional development and behavior. 

LeBlanc and Matson (1995) analyzed two outcomes in this domain and found a statistically significant positive 
effect favoring the intervention on one outcome but no significant difference between the groups on the other 
outcome. However, there was insufficient information for the WWC to calculate effect sizes or to verify the signifi- 
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cance tests conducted by the authors. As a result, the information from this study does not contribute to the rating 
of the evidence of effectiveness for this WWC report. 

Thus, for the sociai-emotionai deveiopment and behavior domain, there are two studies with statistically significant 
positive effects, one study failing to show evidence of an effect (positive or negative), and no studies showing a 
statisticaliy significant or substantively important negative effect. This resuits in a rating of positive effects, with 
a smali extent of evidence. 


Table 4. Rating of effectiveness and extent of evidence for the sociai-emotionai deveiopment and 


behavior domain 

Rating of effectiveness 

Criteria met 

Positive effects 

Strong evidence of a positive 
effect with no overriding contrary 
evidence. 

The review of social skills training in the social-emotional development and behavior domain had two studies 
showing statisticaiiy significant positive effects and no studies showing statistically significant or substantively 
important negative effects. 

Extent of evidence 

Criteria met 

Small 

The review of social skills training in the social-emotional development and behavior domain was based on two 
studies that included two schools and 103 children. 
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Appendix A.1: Research detaiis for Ferentino (1991) 

Ferentino, S. C. (1991). Teaching social skills to preschool children in a special education program. 


Dissertation Abstracts Internationai, 52(08B), 223-4490. 

Tabie A1. Summary of findings Meets WWC evidence standards without reservations 




Study findings 

Outcome domain 

Sample size 

Average improvement index 
(percentile points) 

Statistically significant 

Cognition 

8 classrooms/65 children 

+7 

No 

Social-emotional development 
and behavior 

8 classrooms/65 children 

+9 

Yes 


Setting The study was conducted in a special education school in a suburban metropolitan area of the 
northeastern United States. 


Study sample The eligibility criteria for this study included (a) the ability of the child to function in a class of 
eight to nine children and (b) parental consent to participate. Given these eligibility criteria, 100 
participants in 12 classrooms were eligible from a population of 177 preschool children In a 
special education school. There were two other children in the school intervention group (S) 
that were dropped from the study as outliers. Nearly all participants had speech and language 
impairments; 25% had various other primary disabilities. 

The 12 classrooms in the study were randomly assigned to one of three conditions: 

(a) Four classrooms (n = 32 children) were assigned to a social skills training program to be 
administered in the school (S), 

(b) Four classrooms (n = 33 children) were assigned to a social skills training program that 
would be implemented in both the school and at home (S + H), and 

(c) Four classrooms (n = 33 children) were assigned to a waiting-list comparison group (C). 

For the purpose of this WWC report, the evidence of the social skills training program is identi- 
fied by comparing the school-only group (S) against the waiting-list comparison group (C). Addi- 
tional comparisons of the remaining groups are presented in Appendices D.2 and D.3. 


InterVGntion The classrooms receiving the social skills training program (both the S and the S + H groups) 

group used the “My Friends and Me” curriculum. The program uses group activities and materials 

intended to enhance the personal identity and social development of preschool children. The 
following materials are included: an activity manual; hand puppets; magnets in geometric, 
human, and doll shapes; activity pictures of a classroom, a city, a single-family home, and a 
shopping center; an illustrated story book; song cards and recorded songs; an activity board 
and liquid-chalk pens; and 30 take-home activity sheets (for the school and at-home group). 

In the school-only group, children participated in 30 half-hour sessions conducted by their 
teacher over the course of four months. In the school and at-home group, children partici- 
pated in 15 half-hour sessions conducted by their teacher and 15 additional sessions at home 
conducted by their parents. 
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Comparison 

group 

Outcomes and 
measurement 


Support for 
implementation 


Children in the comparison group (C) participated in special arts and crafts projects for the 30 
half-hour sessions. These children may have received incidental social skills training. 

The primary outcomes in this study were in the cognition and socio-emotional development 
and behavior domains. 

One outcome was assessed in the cognition domain: 

(a) the Comprehension subtest of the Wechsler Preschool and Primary Scale of 
Intelligence-Revised (WPPSI-R). 

Five outcomes were assessed in the socio-emotional development and behavior domain: 

(a) Vineland Adaptive Behavior Scales Classroom Edition-Socialization domain 
(VABS-C); 

(b) Vineland Adaptive Behavior Scales Interview Edition, Survey Form-Socialization domain 
(VABS-I); 

(c) the Child Behavior Checklist for Ages 4-16 and 2-3, teachers’ ratings (CBCLT 4-16; 
CBCLT 2-3); 

(d) the Child Behavior Checklist for Ages 4-16 and 2-3, parents’ ratings (CBCLP 4-16; 
CBCLP 2-3); 

(e) the Face Recognition subtest of the Kaufman Assessment Battery for Children (K-ABC). 
For a more detailed description of these outcome measures, see Appendix B.® 

Two workshops were held to train teachers and parents (for the school and at-home group) 
on the “My Friends and Me” intervention. 
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Appendix A.2: Research detaiis for Gugiieimo and Tryon (2001) 

Guglielmo, H. M., & Tryon, G. S. (2001). Social skill training in an integrated preschool program. School 
Psychology Quarterly, 16(2), 158-175. 

Tabie A2. Summary of findings Meets WWC evidence standards without reservations 


study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Social-emotional development 6 classrooms/38 children +26 Yes 

and behavior 


Setting 

The study was conducted in a publicly-funded, privately-operated preschool in New York state. 

Study sample 

A total of nine integrated classrooms containing 58 children with developmental delays who 
qualified for special education participated in this study.® Three intact classrooms were ran- 
domly assigned to each of three arms in this study (and the analysis was conducted on the 
eligible sample of students with developmental delays). Group A (n = 19 eligible children) 
received social skills training supplemented by classroom reinforcement of target behaviors. 
Group B (n = 19 eligible children) did not receive social skills training but did receive classroom 
reinforcement of target behaviors. Group C (n = 20 eligible children) did not receive either 
social skills training or classroom reinforcement of target behaviors. For the purpose of this 
WWC report, the evidence of the social skills training program is identified by comparing the 
children in the social skills training supplemented by classroom reinforcement group (Group A) 
against the children who did not receive social skills training but did receive classroom rein- 
forcement of target behaviors (Group B).^° Additional contrasts for Group A against Group C are 
presented in Appendix D.3. 

Intervention 

group 

Children In the three classrooms In Group A received social skills training using the “Taking 
Part: Introducing Social Skills to Children” program, coupled with classroom reinforcement of 
the behaviors targeted by the training: “sharing” and “being in a group.” During social skills 
training, children with developmental delays were instructed on how to join a group and to 
share with peers. Instructors modeled the activities for the children using puppets and a short 
skit. Following this activity, children practiced the sharing skills with their peers. Children 
were given specific instructions on behaviors in which to engage, including establishing eye 
contact, tapping children on the shoulder as a means to gain attention, and asking to play 
with others. Children in Groups A and B received classroom reinforcement of target behav- 
iors, which included continuous verbal acknowledgment of positive behaviors and tangible 
rewards. The intervention lasted for approximately 20 to 30 minutes each day for a total of 
eight days.'''' 

Comparison 

group 

Children In the comparison group (C) did not receive any social skills training but did receive 
classroom reinforcement of target behaviors. 
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Outcomes and 
measurement 


Support for 
implementation 


There were two primary outcomes in this study in the social-emotional development and 
behavior domain. The frequency of each of the two “sharing” and “being in a group” behaviors 
was assessed through direct observations of children. For a more detailed description of these 
outcome measures, see Appendix B. 

A scripted example lesson plan for social skills training was presented as an appendix in the 
study. The first author taught two one-hour training sessions on modeling and role-playing for 
classroom reinforcement of target behaviors to the teachers and teacher assistants. 
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Appendix A.3: Research detaiis for LeBianc and Matson (1995) 

LeBlanc, L. A., & Matson, J. L. (1995). A social skills training program for preschoolers with develop- 
mental delays: Generalization and social validity. Behavior Modification, 19(2), 234-246. 

Tabie A3. Summary of findings Meets WWC evidence standards without reservations 


study findings 
Average improvement index 

Outcome domain Sample size (percentile points) Statistically significant 


Social-emotional development 6 classrooms/32 children na na 

and behavior 


na = not applicable 


Setting 

The study was conducted in a preschool for children with developmental disabilities. 

Study sample 

Thirty-two children in six classrooms participated in this study. The children had mild to moderate 
developmental delays, and many also were physically handicapped. Six intact classrooms of 
children were randomly assigned to one of two conditions. Children in classrooms assigned 
to the intervention group (n = 16 children) received social skills training, while children in class- 
rooms assigned to the comparison group (n = 16 children) received the business-as-usual 
classroom experience. 

Intervention 

group 

Children in the intervention group received two one-hour sessions of social skills training each 
week, for a total of 12 sessions over six weeks. The activities in the training were designed to 
promote social skills and social play. Each session was broken into two phases. During Phase 1 
(approximately 15 minutes), children were instructed on target behaviors, including greeting, asking 
to see a toy, initiating play, and showing a toy. Therapists modeled the target behavior using a pup- 
pet, the children modeled the behavior following this initial presentation with a puppet, and then 
the children modeled the behavior with another child. During Phase 2 (approximately 45 minutes), 
children engaged in play and received verbal and edible reinforcements for engaging in the target 
behavior. Children who engaged In inappropriate behaviors were prompted to perform an opposite 
positive behavior. For example. If a child inappropriately took a toy, the corresponding opposite 
positive behavior was to return the toy and to ask to see it. If the child refused to conduct the 
opposite positive behavior, they were placed in a one-minute time-out. 

Comparison 

group 

Children in the comparison group participated in regular classroom activities and received 
a reward for participating in the data collection for the study. They did not receive any social 
skills training or reinforcement of target behaviors. 

Outcomes and 
measurement 

The study examined two outcomes in the socio-emotional development and behavior domain. 
The frequency of “appropriate” and “inappropriate” behaviors was assessed through direct 
observations of children during a semi-structured play session at pretest and at posttest.^^ por 
a more detailed description of these outcome measures, see Appendix B. 

Support for 
implementation 

The staff that implemented the intervention was comprised of therapists and assistant therapists. 
No information was provided about training. 
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Appendix B: Outcome measures for each domain 


Cognition 

Comprehension subtest of the Wechsler 
Preschool and Primary Scale of 
Intelligence-Revised (WPPSI-R) 

This subtest is part of the Verbal Scaie on the WPPSI-R. This test assesses Intellectual tunctioning of children 
between the ages of 3 and 7. Questions about real-lite situations (requiring social judgment and reasoning) 
are read to the child, and responses are coded tor the quality of the answer and the degree of generalization 
presented (as cited in Ferentino, 1991). 

Social-emotional development 
and behavior 

Child Behavior Checklist for Ages 4-16 
and 2-3, parents’ ratings (CBCLP 4-16; 
CBCLP2-3) 

This instrument captures parent reports of child behavioral problems as well as competencies. The instrument 
assesses both internalizing behaviors (such as depression) and externalizing behaviors (such as aggression) 

(as cited in Ferentino, 1991). 

Child Behavior Checklist for Ages 4-16 
and 2-3, teachers’ ratings (CBCLT 4-16; 
CBCLT2-3) 

This instrument captures teacher reports of child behavioral problems as well as competencies. The instrument 
assesses both internalizing behaviors (such as depression) and externalizing behaviors (such as aggression) 

(as cited in Ferentino, 1991). 

Face Recognition subtest of the Kaufman 
Assessment Battery for Children (K-ABC) 

This assessment requires children to look at one or two faces and then carefully identify that same face in a 
group photograph that shows the taces in ditferent poses (as cited in Ferentino, 1991). 

Frequency of "appropriate ” behaviors 

Five-minute observation segments were sampled trom semi-structured play sessions that occurred during 
pretest and posttest assessment periods. The count ot the appropriate social behaviors “such as saying ‘hi’, 
asking to play, smiling, etc.” (p. 241) served as the outcome of interest in the LeBlanc and Matson (1995) study. 

Frequency of "being in a group” 
behaviors 

During three 20-minute observation periods, the frequencies of target behaviors were recorded in 1 5-second 
blocks for each participant in the study. The number of times that a participant remained in close proximity of 
one other student during this period serves as an outcome of interest for the Guglielmo and Tryon (2001) study. 

Frequency of "inappropriate” behaviors 

Five-minute observation segments were sampled trom semi-structured play sessions that occurred during 
pretest and posttest assessment periods. The count ot the inappropriate social behaviors “such as aggression, 
crying, bullying, etc.” (p. 241) served as the outcome of interest in the LeBlanc and Matson (1995) study. 

Frequency of "sharing” behaviors 

During three 20-minute observation periods, the frequencies of target behaviors were recorded in 15-second 
blocks tor each participant in the study. The number ot times that a participant jointly used a toy or materials with 
another student during this period serves as an outcome ot interest for the Guglielmo and Tryon (2001) study. 

Socialization domain of the Vineland 
Adaptive Behavior Scales, Classroom 
Edition (VABS-C) 

This assessment is made up of the three subdomain assessments: Coping Skills, Interpersonal Relationships, 
and Play and Leisure Time, all administered via a questionnaire to the student's teacher. The scores on these 
subscales are combined to produce an overall Socialization domain score (as rated by the teacher) (as cited in 
Ferentino, 1991). 

Coping Skills subdomain of the 
Socialization domain of the VABS-C 

This subdomain assessment contains 18 items targeting impulse control, responsibility, and manners (as cited 
in Ferentino, 1991). 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-C 

This subdomain assessment contains 17 items targeting appropriately expressing emotions (as cited in 
Ferentino, 1991). 

Play and Leisure Time subdomain 
of the Socialization domain of the 
VABS-C 

This subdomain assessment contains 18 items targeting sharing, playing with others, and cooperating (as cited 
in Ferentino, 1991). 

Socialization domain of the Vineland 
Adaptive Behavior Scales, Interview 
Edition (VABS-I) 

This assessment is made up of the three subdomain assessments: Coping Skills, Interpersonal Relationships, 
and Play and Leisure Time, typically administered via interview with the student’s parent. The scores on these 
subscales are combined to produce an overall Socialization domain score (as rated by the parent). In the 
Ferentino (1991) study, the parent data were obtained via a mailed questionnaire. 

Coping Skills subdomain of the 
Socialization domain of the VABS-I 

This subdomain assessment contains 18 items targeting impulse control, responsibility, and manners (as cited 
in Ferentino, 1991). 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-I 

This subdomain assessment contains 28 items targeting appropriately expressing emotions (as cited in 
Ferentino, 1991). 
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Appendix C.1: Findings inciuded in the rating for the cognition domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Ferentino, 1991^ 

Comprehension subtest 
of the Wechsler Preschool 

Preschoolers 

8 classrooms/ 
65 children 

9.35 

(2.43) 

8,76 

(3.61) 

0.59 

0.19 

+7 

0.14 


and Primary Scale of 
Intelligence — Revised 
(WPPSI-R) 

Domain average for cognition (Ferentino, 1991) 0.19 +7 Not 

statistically 

significant 


Domain average for cognition across all studies 0.19 +7 na 


Table Notes: For mean difference, effect size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. The WWC-computed average effect size is a simpie average rounded 
to two decimai piaces; the average improvement index is caicuiated from the average effect size. The statisticai significance of the study's domain average was determined by the 
WWC. na = not appiicabie. 

“ For Ferentino (1 991 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy significant. The 
p-vaiue presented here was reported in the originai study. The evidence of the social skills training program is identified by comparing the schooi-oniy group (S) against the waiting- 
iist comparison group (C). Additionai contrasts for aii groups are presented in Appendix D.2. The WWC caicuiated the social skills training group mean by adding the difference-in- 
differences adjusted estimate of the average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison 
group posttests means. Piease see the WWC Handbooktor more information. 
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Appendix C.2: Findings inciuded in the rating for the sociai-emotionai deveiopment and behavior domain 




Mean 

(standard deviation) 

WWC calculations 


Study 

Outcome measure sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Ferentino, 1991^ 

Child Behavior Checklist for Preschoolers 

Ages 4-16 and 2-3, parents’ 
ratings (CBCLP 4-16; CBCLP 
2-3) 

8 classrooms/ 
65 children 

49.13 

(11.17) 

47,55 

(13.01) 

1.58 

0.13 

+5 

0.18 

Child Behavior Checklist for Ages Preschoolers 
4-1 6 and 2-3, teachers’ ratings 
(CBCLT4-16; CBCLT2-3) 

8 classrooms/ 
65 children 

43.75 

(9.77) 

45.79 

(11.63) 

-2,04 

-0.19 

-7 

0.96 

Face Recognition subtest of the Preschoolers 

Kaufman Assessment Battery for 
Children (K-ABC) 

8 classrooms/ 
65 children 

9.73 

(2.56) 

11.06 

(2.94) 

-1,33 

-0.48 

-18 

0.26 

Socialization domain of the Preschoolers 

Vineland Adaptive Behavior 
Scales, Classroom Edition 
(VABS-C) 

8 classrooms/ 
65 children 

50.44 

(14.49) 

32,58 

(7.73) 

17.86 

1.53 

+44 

0.00 

Socialization domain of the Preschoolers 

Vineland Adaptive Behavior 
Scales, Interview Edition (VABS-I) 

8 classrooms/ 
65 children 

65.76 

(23.46) 

62.05 

(21,93) 

3,71 

0.16 

+6 

0.81 

Domain average for social-emotional development and behavior (Ferentino, 1991) 


0.23 

+9 

Statistically 

significant 

Guglielmo & Tryon, 2001'’ 

Frequency of “being in a group ’’ Preschoolers 
behaviors 

6 classrooms/ 
38 children 

10.65 

(4.19) 

9.58 

(3.22) 

1.07 

0,28 

+11 

>0.05 

Frequency of “sharing” behaviors Preschoolers 

6 classrooms/ 
38 children 

5.69 

(2.55) 

2.76 

(2.55) 

2,93 

1.12 

+37 

<0.05 

Domain average for social-emotional development and behavior (Guglielmo & Tryon, 2001) 


0.70 

+26 

Statistically 

significant 

LeBlanc & Matson, 1995” 

Frequency of “appropriate ” Preschoolers 

social behaviors 

6 classrooms/ 
32 children 

nr 

nr 

nr 

na 

na 

na 

Frequency of “inappropriate ” Preschoolers 

social behaviors 

6 classrooms/ 
32 children 

nr 

nr 

nr 

na 

na 

na 

Domain average for social-emotional development and behavior (LeBlanc & Matson, 1995) 


na 

na 

na 

Domain average for social-emotional development and behavior across all studies 


0.47 

+18 

na 


Table Notes; For mean difference, effect size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. The WWC-computed average effect size is a simpie average rounded 
to two decimai piaces; the average improvement index is caicuiated from the average effect size. The statisticai significance of each study’s domain average was determined by 
the WWC. na = not appiicabie. nr = not reported. 

“ For Ferentino (1 991 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy significant. The 
p-vaiues presented here were reported in the originai study. The evidence of the social skills training program is identified by comparing the schooi-oniy group (S) against the waiting- 
iist comparison group (C). Additionai contrasts for aii groups are presented in Appendix D.2. The WWC caicuiated the social skills training group mean by adding the difference-in- 
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differences adjusted estimate of the average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison 
group posttests means. Piease see the WWC Handbooktor more information. 

For Gugiieimo and Tryon (2001 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy 
significant. The p-vaiues presented here were reported in the originai study. The evidence of the social skills training program is identified by comparing the chiidren in the social skills 
training suppiemented by ciassroom reinforcement (Group A) against the chiidren not receiving social skills training but receiving ciassroom reinforcement of target behaviors (Group 
B). By comparing Group A against Group B, the effects of ciassroom reinforcement are differenced out of the comparison, and therefore, the observed differences in outcomes can 
be attributed to social skills training. Additionai contrasts for Group A against the no-intervention comparison group (Group C) are presented in Appendix D.3. The WWC caicuiated the 
social skills training group mean by adding the difference-in-differences adjusted estimate of the average impact of the program (i.e., difference in mean gains between the interven- 
tion and comparison groups) to the unadjusted comparison group posttests means. Piease see the WWC Handbooktor more information. 

' For LeBianc and Matson (1 995), insufficient information was provided to caicuiate effect sizes and statisticai significance. The authors did report F-test statistics from an anaiysis of 
covariance (ANCOVA) for each outcome, but the authors’ description of the modei suggests that the anaiysis was actuaiiy a repeated-measures anaiysis of variance (ANOVA). Given the 
uncertainty in the actuai anaiytic modei performed, we do not compute effect sizes from the reported statistics. As a resuit, this study does not contribute information to the domain 
average for socio-emotionai deveiopment and behavior. The authors aiso presented resuits from a generaiization assessment 7-1 0 days after the posttest; however, these contrasts 
were not considered eiigibie to meet WWC standards with or without reservations. 
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Appendix D.1: Summary of subscaie findings for the sociai-emotionai deveiopment and behavior domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Ferentino, 1991“ 

Coping Skills subdomain of the 
Socialization domain of the VABS-C 

Preschoolers 

8 classrooms/ 
65 children 

51.72 

(11.01) 

30.12 

(7.36) 

21.60 

2.29 

-f49 

0.00 

Coping Skills subdomain of the 
Socialization domain of the VABS-I 

Preschoolers 

8 classrooms/ 
65 children 

67.73 

(19.07) 

68.33 

(16.29) 

-0,60 

-0.03 

-1 

0.12 

Interpersonal Relationships subdomain 
of the Socialization domain of the 
VABS-C 

Preschoolers 

8 classrooms/ 
65 children 

55.61 

(32.08) 

28.94 

(10.77) 

26.67 

1.11 

-f37 

0.00 

Interpersonal Relationships subdomain 
of the Socialization domain of the 
VABS-I 

Preschoolers 

8 classrooms/ 
65 children 

73.32 

(35.74) 

62.81 

(36.13) 

10.51 

0.29 

-fII 

0.69 

Play and Leisure Time subdomain of 
the Socialization domain of the VABS-C 

Preschoolers 

8 classrooms/ 
65 children 

45,58 

(10.97) 

41.06 

(9.30) 

4,52 

0.44 

-f17 

0,07 


Table Notes: For mean difference, effect size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. VABS-C = Vineiand Adaptive Behavior Scaies, Ciassroom Edition. 
VABS-i = Vineiand Adaptive Behavior Scaies, interview Edition. 

“ For Ferentino (1 991 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy significant. The 
p-vaiues presented here were reported in the originai study. The evidence of the social skills training program is identified by comparing the schooi-oniy group (S) against the waiting- 
iist comparison group (C). Additionai contrasts for aii groups are presented in Appendix D.3. The WWC caicuiated the social skills training group mean by adding the difference-in- 
differences adjusted estimate of the average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison 
group posttests means. Piease see the WWC Handbooktor more information. 
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Appendix D.2: Summary of aiternate contrasts for the cognition domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Ferentino, 1991^ 

Comprehension subtest of 
the Wechsler Preschool and 
Primary Scale of Intelligence — 
Revised (WPPSI-R) 

School (S) vs. 
School -F Home 
(S + H) 

8 classrooms/ 
65 children 

10.35 

(2.43) 

9.88 

(3.35) 

0.47 

0.16 

-f6 

0.43 

Comprehension subtest of 
the WPPSI-R 

School -F Home 
(S -F H) vs. 

8 classrooms/ 
66 children 

8.88 

(3.35) 

8.76 

(3.61) 

0.12 

0.03 

-fI 

0.48 


Comparison (C) 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for ali students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. 

® For Ferentino (1 991 ), corrections for ciustering and multipie comparisons were needed but did not affect whether any of the contrasts were found to be statistically significant. The 
p-values presented here were reported in the original study. The evidence of the social skills training program is identified by comparing the school-only group (S) against the waiting- 
list comparison group (C) and is included in Appendix C.1 . The WWC calculated the social skills training group mean by adding the difference-in-differences adjusted estimate of the 
average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group posttests means. Please see the 
WWC Handbooktor more information. This appendix contains the alternate contrasts possible in this design. 
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Appendix D.3: Summary of aiternate contrasts for the sociai-emotionai deveiopment and behavior domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Ferentino, 1991“ 

Child Behavior Checklist for Ages 
4-16 and 2-3, parents’ ratings 
(CBCLP 4-16; CBCLP 2-3) 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

56.57 

(11.17) 

55.46 

(11.25) 

1.11 

0.10 

+4 

0.87 

Child Behavior Checklist for Ages 
4-16 and 2-3, teachers’ ratings 
(CBCLT4-16; CBCLT2-3) 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

42.51 

(9.77) 

44.52 

(9.12) 

-2.01 

-0.21 

-8 

0.83 

Face Recognition subtest of the 
Kaufman Assessment Battery 
for Children (K-ABC) 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

8.94 

(2.56) 

10.33 

(2.92) 

-1.39 

-0.50 

-19 

0.58 

Socialization domain of the 
Vineland Adaptive Behavior 
Scales, Classroom Edition 
(VABS-C) 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

53.38 

(14.49) 

48.39 

(14.52) 

4.99 

0.34 

+13 

0.30 

Socialization domain of the 
Vineland Adaptive Behavior 
Scales, Interview Edition 
(VABS-I) 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

64.56 

(23.46) 

55.27 

(16.34) 

9.29 

0.46 

+18 

0.39 

CBCLP 4-1 6; CBCLP 2-3 

School + Home 
(S + H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

48.02 

(11.25) 

47.55 

(13.01) 

0.47 

0.04 

+2 

0.13 

CBCLT4-16; CBCLT2-3 

School + Home 
(S + H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

45.76 

(9.12) 

45.79 

(11.63) 

-0.03 

0.00 

0 

0.79 

Face Recognition subtest of the 
K-ABC 

School + Home 
(S + H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

11.12 

(2.92) 

11.06 

(2.94) 

0.06 

0.02 

+1 

0.57 

Socialization domain of the 
VABS-C 

School + Home 
(S + H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

45.45 

(14.52) 

32.58 

(7.73) 

12.87 

1.09 

+36 

0.00 

Socialization domain of the 
VABS-I 

School + Home 
(S + H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

56.47 

(16.34) 

62.05 

(21.93) 

-5.58 

-0.29 

-11 

0.27 

Coping Skills subdomain of 
the Socialization domain of the 
VABS-C 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

57.90 

(11.01) 

48.70 

(13.49) 

9.20 

0.74 

+27 

0.06 

Coping Skills subdomain of 
the Socialization domain of the 
VABS-I 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

68.76 

(19.07) 

64.12 

(13.67) 

4.64 

0.28 

+11 

0.61 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-C 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

59.73 

(32.08) 

51.79 

(27.63) 

7.94 

0.26 

+10 

0.21 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-I 

School (S) vs. 
School + Home 
(S + H) 

8 classrooms/ 
65 children 

72.96 

(35.74) 

58.35 

(23.07) 

14.61 

0.48 

+18 

0.17 
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Play and Leisure Time 
subdomain of the Socialization 
domain of the VABS-C 

School (S) vs. 
School -F Home 
(S + H) 

8 classrooms/ 
65 children 

42.79 

(10.97) 

47.73 

(14.48) 

-4.94 

-0.38 

-15 

0.32 

Coping Skills subdomain of 
the Socialization domain of the 
VABS-C 

School -F Home 
(S -F H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

42.52 

(13.49) 

30.12 

(7.36) 

12.40 

1.13 

-f37 

0.00 

Coping Skills subdomain of 
the Socialization domain of the 
VABS-I 

School -F Home 
(S -F H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

63.09 

(13.67) 

68.33 

(16.29) 

-5.24 

-0.34 

-13 

0.28 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-C 

School -F Home 
(S -F H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

47.67 

(27.63) 

28.94 

(10.77) 

18.73 

0.88 

-f31 

0.00 

Interpersonal Relationships 
subdomain of the Socialization 
domain of the VABS-I 

School -F Home 
(S -F H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

58.71 

(23.07) 

62.81 

(36.13) 

-4.10 

-0.13 

-5 

0.35 

Play and Leisure Time 
subdomain of the Socialization 
domain of the VABS-C 

School -F Home 
(S -F H) vs. 
Comparison (C) 

8 classrooms/ 
66 children 

50.52 

(14.48) 

41.06 

(9.30) 

9.46 

0.77 

-f28 

0.01 


Guglielmo & Tryon, 2001'’ 


Frequency of “being in a group” 
behaviors 

Training and 
classroom (Group 
A) vs. Comparison 
(Group C) 

6 classrooms/ 
39 children 

9.19 

(4.19) 

2.75 

(1.91) 

6.44 

1.95 

-f47 

<0.05 

Frequency of “sharing” 
behaviors 

Training and 
classroom (Group 
A) vs. Comparison 
(Group C) 

6 classrooms/ 
39 children 

4.98 

(2.55) 

2.68 

(2.33) 

2.30 

0.92 

-f32 

<0.05 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. 

® For Ferentino (1 991 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy significant. The 
p-vaiues presented here were reported in the originai study. The evidence of the social skills training program is identified by comparing the schooi-oniy group (S) against the waiting- 
iist comparison group (C) and is shown in Appendix C.2. The WWC caicuiated the social skills training group mean by adding the difference-in-differences adjusted estimate of the 
average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group posttests means. Piease see the 
WWC Handbooktor more information. This appendix contains the aiternate contrasts possibie in this design. 

For Gugiieimo and Tryon (2001 ), corrections for ciustering and muitipie comparisons were needed but did not affect whether any of the contrasts were found to be statisticaiiy 
significant. The p-vaiues presented here were reported in the originai study. The evidence of the social skills training program is identified by comparing the chiidren in the social skills 
training suppiemented by ciassroom reinforcement (Group A) against the chiidren not receiving social skills training but receiving ciassroom reinforcement of target behaviors (Group 
B) and is shown in Appendix C.2. By comparing Group A against Group B, the effects of ciassroom reinforcement are differenced out of the comparison, and therefore, the observed 
differences in outcomes can be attributed to social skills training. The WWG caicuiated the social skills training group mean by adding the difference-in-differences adjusted estimate 
of the average impact of the program (i.e., difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group posttests means. Piease 
see the WWC Handbook for more information. This appendix contains the aiternate contrasts possibie in this design. 
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Endnotes 

■' Social skills training does not have a single developer or official description. The descriptive Information for this program was adapted 
from publicly available sources: descriptions of this practice (see the websites listed under Program Information) and research articles 
(Ferentino, 1 991 ; Gugllelmo & Tryon, 2001 ; LeBlanc & Matson, 1 995). Further verification of the accuracy of the descriptive information for 
this program is beyond the scope of this review. The literature search reflects documents publicly available by August 201 1 . 

^ The studies in this report were reviewed using WWC Evidence Standards, version 2.1 , as described in the Early Childhood Educa- 
tion Interventions for Children with Disabilities review protocol, version 2.1. The evidence presented in this report is based on available 
research. Findings and conclusions may change as new research becomes available. 

® For criteria used in the determination of the rating of effectiveness and extent of evidence, see the WWC Rating Criteria on p. 26. 
These improvement index numbers show the average and range of student-level Improvement Indices for all findings across the 
studies. The WWC review of Interventions for Early Childhood Education Interventions for Children with Disabilities addresses student 
outcomes In seven domains: cognition, communication/language competencies, literacy, mathematics achievement, social-emotional 
development and behavior, functional abilities, and physical well-being. Table 1 includes results only for cognition and social-emo- 
tional development and behavior, as these were the only domains for which outcomes were assessed in the three studies that meet 
evidence standards. 

A third arm of the study (four classrooms, n = 33 children) supplemented the classroom-based implementation with an at-home addi- 
tional element of training. Contrasts against this arm of the intervention are presented in Appendices D.2 and D.3. 

® A third arm of the study (three classrooms, n = 20 children) did not receive any intervention at all. Contrasts that compare the group 
receiving social skills training supplemented by a reinforcement of targeted behaviors against this no-intervention arm are included In 
Appendix D.3. 

® The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for 
clustering within classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical 
significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the cases of Ferentino (1991) and Gugllelmo and Tryon (2001), corrections for 
clustering and multiple comparisons were needed, so the significance levels may differ from those reported in the original studies. 

For LeBlanc and Matson (1995), Insufficient information was provided to calculate effect sizes and statistical significance. The authors 
did report F-test statistics from an analysis of covariance (ANCOVA) for each outcome, but the authors’ description of the model 
suggests that the analysis was actually a repeated-measures analysis of variance (ANOVA). Given the uncertainty in the actual analytic 
model performed, we do not compute effect sizes from the reported statistics, and thus, the evidence from this study is not incorporated 
into the effectiveness ratings. 

^ According to the WWC Procedures and Standards Handbook, Appendix G, Ferentino (1991) is characterized as having statistically 
significant positive effects because it meets the following criteria: “Univariate statistical tests are reported for each outcome measure, 
and the effect for at least one measure within the domain is positive and statistically significant, and no effects are negative and statis- 
tically significant, accounting for clustering and multiple comparisons.” There was one substantively important negative effect found in 
this study, but because it was not statistically significant, the study Is determined to have a statistically significant positive effect. 

® The authors also reported results for three subtests (Coping Skills, Interpersonal Relationships, and Play and Leisure Time) of the 
Vineland Adaptive Behavior Scales (the subtest scores are pooled to generate the Socialization domain score). These results are pre- 
sented In Appendix D.1 . The results from the Vineland Adaptive Behavior Scales Interview Edition, Survey Form, Socialization domain 
(VABS-I)-Play and Leisure Time subdomain scale are not Included in this WWC report, as this subdomain was not shown to be suf- 
ficiently reliable to meet WWC standards. In addition, the authors presented results for sociometric scales and rankings of children 
(based on peer and teacher nomination); however, these outcomes do not align with the domains used in this topic area and are not 
included In the Intervention report. For further details about the outcomes included In the Early Childhood Education Interventions for 
Children with Disabilities topic review, please see the Early Childhood Education Interventions for Children with Disabilities Review 
Protocol. 

® Originally, 69 children were assigned to one of the three Intervention conditions, but the number assigned to each was not reported. 
For the purpose of assessing the study’s rating, it was assumed that an equal number of children (n = 23) were assigned to each Inter- 
vention condition. 

By comparing Group A against Group B, the effects of classroom reinforcement are effectively removed from the comparison, and 
therefore, the observed differences In outcomes can be attributed to social skills training. 
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After eight days of the intervention, children in Group A stopped receiving social skills training, and children in Group B began 
receiving the social skills training intervention and were assessed following this crossover. In order to obtain an uncontaminated esti- 
mate of the effect of social skills training, impacts presented in this WWC report include only contrasts assessed after the initial eight 
days of intervention (prior to the intervention crossover and subsequent follow-up assessment). 

''2 In addition to the posttest assessment, the authors also conducted a generalization assessment. While overall attrition for this 
contrast was presented, it was unclear if there was differential attrition across the two intervention groups, which would have resulted 
in a high attrition rate for this contrast. Under the conservative assumption that there was high differential attrition, these contrasts are 
unable to meet evidence standards (as the inferential tests did not adjust for baseline differences), and as such, they are not included 
in the report. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, February). 

Early Childhood Education Interventions for Children with Disabilities intervention report: Sociai skilis training. 
Retrieved from http://whatworks.ed.gov. 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 


Study rating 

Criteria 

Meets WWC evidence standards 
without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a weii-implemented RCT. 

Meets WWC evidence standards 
with reservations 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 
attrition that has established equivaience of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statisticaiiy significant positive effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 

of sfudies show indeterminafe effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show 
a statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 
The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 
The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students 
in a class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 

Single-case design 
Standard deviation 


Statistical significance 


Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 26. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 26. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 
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